Varying The Exposure Of A Digital Image By Region

ABSTRACT

Methods and apparatus for creating a digital image in which exposure varies by region of the image are described. The image may be created from first and second images. In one embodiment, a method comprises writing pixels of the first image that are within a first region of a space to pixel locations in a memory that correspond with coordinates of the respective first image pixels. A replacement pixel is generated by combining a pixel of the first image that is within the first region with a spatially corresponding pixel of a second image. The replacement pixel is written to a pixel location in the memory that corresponds with the coordinates of the replacement pixel. Pixels of the first image that are within a second region of the space may be written to pixel locations in the memory that correspond with the coordinates of the respective first image pixels.

TECHNICAL FIELD

The present invention is in the field of digital image processing. Morespecifically, the present invention relates to generating a digitalimage in which exposure varies by region of the image.

BACKGROUND

In photography, there is often a need to adjust brightness of differentparts of an image. For example, consider an image captured in the middaysun. If an exposure is selected so that brightly lit objects areaccurately captured, shadowed objects will be very dark and lack detail.On the other hand, if an exposure is selected so that shadowed objectsare accurately captured, the brightly lit objects will appear washedout.

Traditionally, this problem has been solved with a “fill flash”technique. A flash unit illuminates areas that would otherwise be inshadow. One disadvantage of using a fill flash is that it requiresknowledge and training that is beyond what many amateur photographerspossess. Another disadvantage is that it requires a flash attachment forthe camera.

In conventional film photography, this problem has also been solved withdark room techniques, such as “dodging” an area of a print to make itlighter or “burning” an area of a print to make it darker. However, thetraditional dodging and burning techniques are not available to digitalphotographers.

In digital photography, special purpose digital photo manipulationsoftware running on a personal computer can be used to manipulate adigital image. For the casual photographer this approach has severaldisadvantages. The software is expensive, requires powerful hardware,and requires a substantial investment of time to learn how to use thesoftware. Moreover, the casual photographer is accustomed to seeing acaptured image immediately. With special purpose photo manipulationsoftware, there is a significant time delay before the corrected imagemay be seen.

Accordingly, there is a need for methods and apparatus for generating adigital image in which exposure varies by region of the image.

SUMMARY

Methods and apparatus for creating a digital image in which exposure maybe varied by region of the image are described. The image may be createdfrom two or more images and pixels of the images have coordinates in aspace. In one embodiment, pixels of a first one of the images that arewithin a first region of the space are written to pixel locations in adevice that correspond with the coordinates of the respective firstimage pixels. In addition, replacement pixels are generated by combiningeach pixel of the first image that is within the first region with aspatially corresponding pixel in a second one of the images. Eachreplacement pixel has the same coordinates as the pixels from which itis generated. Further, each of the replacement pixels may be written toa pixel location in the device that corresponds with the coordinates ofthe replacement pixel.

In another embodiment, pixels of a first one of the images that arewithin a second region of the space are written to pixel locations inthe device that correspond with the coordinates of the respective firstimage pixels. In yet another embodiment, pixels of a second one of theimages that are within the second region are written to pixel locationsin the device that correspond with the coordinates of the respectivesecond image pixels, where the second region pixels of the second imageare written to the device after the second region pixels of the firstimage are written to the device. In still another embodiment, pixels ofa first one of the images that are within the second region arediscarded.

Moreover, in one embodiment, a first image pixel may be combined with aspatially corresponding second image pixel in the second image togenerate a replacement pixel by adding the respective pixels. In analternative embodiment, a first image pixel may be combined with aspatially corresponding second image pixel in the second image togenerate a replacement pixel by subtracting one pixel from the other.Further, in various embodiments, the device is a memory.

In a further embodiment, a digital image in which exposure may be variedby region of the image is created. The image may be created from two ormore images and pixels of the images have coordinates in a space. Pixelsof a first one of the images that are within a first region of the spaceare written to pixel locations in a device that correspond with thecoordinates of the respective first image pixels. In addition, pixels ofa second one of the images that are within a second region of the spaceare written to pixel locations in the device that correspond with thecoordinates of the respective second image pixels. The second image hasan exposure period that is distinct from the exposure period of thefirst image. In one alternative, the device is a memory.

Another embodiment is directed to a display controller. The displaycontroller comprises a memory to store image data, a positiondetermining unit, and a combining unit. The memory has a plurality ofpixel locations for pixels of an image. The pixel locations correspondwith the coordinates of the respective pixels in a space. The positiondetermining unit determines whether a pixel is within one of at least afirst and second region of the space. The combining unit generatesreplacement pixels. The combining unit combines a pixel of a first imagethat is within the first region with a spatially corresponding pixel ofa second image. The replacement pixel has the same coordinates as thepixels from which it is generated. The combining unit stores thereplacement pixel in a pixel location in the memory that correspondswith the coordinates of the replacement pixel.

In one embodiment, the position determining unit stores a first imagepixel that is within the first region in a pixel location in the memorythat corresponds with the coordinates of the first image pixel. In oneembodiment, the position determining unit stores a first image pixelthat is within the second region in a pixel location in the memory thatcorresponds with the coordinates of the pixel. In another embodiment,the position determining unit stores a second image pixel that is withinthe second region in a pixel location in the memory that correspondswith the coordinates of the second image pixel. In yet anotherembodiment, the combining unit reads a first image pixel that is withinthe first region from the memory.

In one embodiment, the combining unit combines a first image pixel witha spatially corresponding second image pixel to generate a replacementpixel by adding the pixels. Alternatively, the combining unit combines afirst image pixel with a spatially corresponding second image pixel togenerate a replacement pixel by subtracting one pixel from the other. Inaddition, in one embodiment the display controller generates areplacement pixel in real-time.

Moreover, one alternative embodiment is directed to a displaycontroller. The display controller comprises a memory and a positiondetermining unit. The memory is employed to store image data. The memoryhas a plurality of pixel locations for pixels of an image. The pixellocations correspond with the coordinates of the respective pixels in aspace. The position determining unit determines whether a pixel iswithin one of at least a first and second region of the space. Theposition determining unit writes pixels of a first image that are withina first region to pixel locations in the memory that correspond with thecoordinates of the respective first image pixels. In addition, thecombining unit writes pixels of a second image that are within thesecond region to pixel locations in the device that correspond with thecoordinates of the respective second image pixels. The second image hasan exposure period that is distinct from the exposure period of thefirst image.

This Summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription. This Summary is not intended to identify key or essentialfeatures of the claimed subject matter, nor is it intended to be used asan aid in determining the scope of the claimed subject matter.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an image, a first prophetical frame of the imagecaptured at a first exposure, the first frame having first and secondregions, and a second prophetical frame having exposures that vary byregion of the image.

FIG. 2 is a diagram of prophetical first and second frames, a space, anda memory having a third frame stored in the memory, each of the frameshaving first and second regions.

FIG. 3 is a flow diagram illustrating methods according to variousembodiments.

FIG. 4A is a diagram of the first and second frames of FIG. 2, the firstand second frames including image data, and the memory of FIG. 2 atfirst and second points in time, the image data of the frames beingstored in the memory, the figure illustrating one example of creating animage having exposures that vary by region.

FIG. 4B is a diagram of the memory of FIG. 2 at the second point intime, the figure illustrating another example of creating an imagehaving exposures that vary by region.

FIG. 5 is a diagram of the first and second frames of FIG. 2, the firstand second frames including image data, and the memory of FIG. 2 atfirst and second points in time, the figure illustrating yet anotherexample of creating an image having exposures that vary by region.

FIG. 6 is a diagram of the first and second frames of FIG. 2, the firstand second frames including image data, and the memory of FIG. 2 atfirst and second points in time, the figure illustrating an additionalexample of creating an image having exposures that vary by region.

FIG. 7 is a simplified block diagram of an exemplary display system forproviding frames that have different exposures in different regions ofthe frame.

Generally, the same reference numbers are used in the drawings and thedescription to refer to the same or like parts, elements, or steps.

DETAILED DESCRIPTION

FIG. 1 illustrates an image 20 of a face lit from the side. As a resultof this lighting, the left side of the face is in shadow. The image 20is preferably more or less stationary, however, this is not critical. Aprophetical frame 22 shows how the image 20 might appear when capturedat a single exposure of, for example, 0.004 seconds. The frame 22includes a first region 24, which includes the shadowed portion, and asecond region 26. FIG. 1 also shows a second frame 28. The second frame28 is a prophetical frame of the image 20 in which exposure varies byregion of the image. For example, the first region 24 of frame 28 has anexposure of 0.008 seconds and the second region 26 has an exposure of0.004 seconds. In the prophetical frame 28, the image 20 appears withoutthe shadow on the left side of the face.

FIG. 2 shows prophetical first and second frames 30, 32 that include thefirst and second regions 24, 26. The frames 30, 32 are alternativelydesignated F′ and F″, respectively. The frames 30, 32 as used below inthis description include image data captured at different times. Theframes 30, 32 may both be created as a result of capturing the sameimage 20, however, the frames may be created from separate captureoperations. For example, the frame 30 may result from capturing theimage 20 at a first point in time, and the frame 32 may result fromcapturing the image 20 at a second point in time. The two frames may becaptured at closely spaced points in time, however, this is notessential. For example, the first and second frames 30, 32 may beseparated by 0.001 seconds, 0.01 seconds, 0.1 seconds, or 1 second.Longer or shorter intervals are also contemplated. In addition, the twoexposures may be of the same or different durations.

Alternative designations are also employed for the first and secondregions 24, 26. In the first frame F′, the first and second regions 24,26 are also respectively designated R′₁ and R′₂. In the second frame F″,the first and second regions 24, 26 are respectively designated R″₁ andR″₂.

The image data in the frames 30, 32 is comprised of a matrix of pixels.Each of the pixels in each of the frames has a location in atwo-dimensional space 34. In other words, the location of each pixel ineach of the frames may be defined by a pair x and y coordinates. While apixel is the smallest complete sample in an image, a pixel may becomprised of multiple components, e.g., individual red, blue, and greencomponents (“RGB”). The data that defines a pixel may thus be comprisedof three distinct datum. For example, three bytes may be used to defineeach of the color components of an RGB pixel.

A memory 36 is also shown in FIG. 2. A portion of this memory 36 isallocated for storing one frame of image data. The memory 36 may includenumerous memory locations. Each memory location stores one or more bitsof data. In this description and in the claims, the term “pixellocation” is used to refer to one or more memory locations within amemory that are used for storing one complete pixel. Using thisterminology, the memory 36 can be said to include numerous pixellocations. Thus, if the RGB pixels of a frame are defined by threebytes, each pixel location may designate three distinct memorylocations, each for storing one byte. The memory locations that comprisea pixel location need not be in any particular location in the memory.For instance, they may be contiguous memory locations, or all of thecomponents of a particular type may be grouped together. What isimportant is that the pixel locations in the memory 36 correspond withthe coordinates of pixels in the space 34. In other words, the spatiallocation of each pixel in each of the frames corresponds with a distinctpixel location in the memory. As mentioned, the spatial location of eachpixel may be expressed with an (x, y) coordinate. Where a pixel locationincludes two or more memory locations, e.g., for storing colorcomponents, the coordinate location of a pixel corresponds with each ofthe two or more locations.

FIG. 2 shows a prophetical frame 38 stored in the memory 36. Because ofthe correspondence between pixel locations in the memory 36 and thecoordinates of each pixel in each of the frames 30, 32, every pixel inthe frame 30 will correspond to both a distinct pixel location in thememory 36 and to a distinct pixel in the frame 32. Similarly, everypixel in the frame 32 will correspond to both a distinct pixel locationin the memory 36 and to a distinct pixel in the frame 30. The frame 38also includes the regions 24 and 26.

The regions 24, 26 may be defined in a variety of ways. The regions 24,26 may be rectangular and, in this case, can be defined by a pair of (x,y) coordinates. However, the first and second regions 24, 26 may takeother shapes, e.g., circular, elliptical, irregular, etc., and may bedefined by any function suitable for defining the desired shape. In FIG.2, the first and second regions are designated R₁ and R₂ in the frame 38in the memory 36. Again, because of the correspondence between pixellocations in the memory 36 and the pixels in the frames, the pixels inregion R′₁ spatially correspond with the pixels in region R″ ₁, as wellas with region R₁ in the memory 36. A similar correspondence also existsfor the regions R′₂, R″₂, and R₂.

Two or more digital images may be combined by summing the data values ofspatially corresponding pixels of the images. Where the pixels arerepresented by multiple components, the frames may be combined bysumming the respective components of spatially corresponding pixels. Inaddition to summing, two or more digital images may be combined bysubtracting one image from the other.

FIG. 3 is a flow diagram of a method 40 that illustrates in a generalmanner several ways that a frame or digital image having exposures thatvary by region may be created. The method 40 may be performed in anyhardware device having a memory of sufficient size to store one frame.It is generally not necessary, however, for the memory to have capacityfor storing more than one frame. In one embodiment, the memory may havecapacity for storing less than one frame. In addition, various methodsaccording to the invention may be performed partially in hardware andpartially in software. With respect to the method 40, assumptions aremade that the first and second regions 24, 26 have been previouslydefined in the space 34, and that one of the two regions has been“specified” as described below. Particular steps of the method 40 may beperformed a pixel at a time. Steps according to various embodiments ofthe invention may be performed in raster order. Alternatively, steps maybe performed on pixels arranged in any desired order.

Referring to FIG. 3, a first frame 30 is stored in the memory (step 42).In one embodiment, the step 42 of storing may store both the first andsecond regions of the first frame in the memory. A second frame isreceived (step 44). The first and second frames may be received a pixelat a time. In one embodiment, a determination is made for each pixel ofthe second frame as to whether the pixel is within the first or secondregions (step 46). Specifically, either the first or second region maybe “specified.” The step 42 then determines whether a particular pixelis within the specified region. Each pixel of the first frame that iswithin the specified region is combined with a spatially correspondingpixel in the second frame (step 48). (Except where noted, the examplesthat follow assume that spatially corresponding pixels of the first andsecond frames are combined by addition.) A replacement pixel isgenerated in step 48 as a result of combining the spatiallycorresponding pixels of the two frames. In order to combine the twopixels in step 48, it may be necessary to read the corresponding pixelof the first frame from the memory (step 52). The two pixels may becombined by summing the two pixels or by subtracting one pixel from theother. The replacement pixel that is generated is assigned the samecoordinates as the two pixels from which it was created. The replacementpixel is stored in a pixel location in the memory that corresponds withthe replacement pixel's coordinates (step 50).

Pixels of the second frame that are not found to be in the specifiedregion may be stored in the memory (step 54) or, alternatively, they maybe discarded (step 56). If pixels of the second frame that are outsideof the specified region are stored in the memory in step 54, they arestored in pixel locations that correspond with the coordinates of therespective pixels. In addition, the pixels of the second frame that areoutside of the specified region are stored in an operation that destroysthe pixel previously stored in the same pixel location. That is, thepixels of the second frame that are located outside of the specifiedregion may be stored in a destructive write operation, which replacespixels of the first frame that are outside of the specified region andthat may have been stored in the same pixel location in step 42.

Several examples further illustrate variations of the method 40. In theexamples, it is assumed that the “specified” region is the first region,i.e., R′₁, R″₁. Referring to FIG. 4A, a first example is shown. A firstframe 30 is stored in the memory 36 at time 1 (step 42). The first frame30 is stored in the memory as frame 38 a. At time 2, the second frame isreceived (step 44). The position of each pixel of the second frame inthe space 34 is evaluated to determine whether or not the particularpixel is within the specified region (step 46). Replacement pixels R₁are generated for pixels within the specified region (step 48). In thisexample, replacement pixels R₁ are generated by adding pixels in thefirst region of the first and second frames, i.e., R′₁+R″₁. Replacementpixels R₁ are also stored in the memory at time 2 (step 50). Inaddition, the pixels R″₂ of the second frame that are not within thespecified region are discarded (step 56). Thus, at time 2 the frame 38 bstored in the memory is comprised of replacement pixels R₁=R′₁+R″₁ inthe specified region, and pixels R′₂ the second region. If the first andsecond frames 30, 32 are both exposed at 0.004 seconds, then the firstregion R₁ in the frame 38 b will have an exposure, assuming the pixelsare combined by summing, of 0.008 seconds, and the second region R′₂ inthe frame 38 b will have an exposure of 0.004 seconds.

As one alternative, the spatially corresponding pixels of the first andsecond frames are combined in the example of the preceding paragraph bysubtraction instead of addition. If the first frame 30 is exposed at0.008 seconds, and the second frame 32 is exposed at 0.004 seconds, thenthe first region in the frame 38 b will have an exposure of 0.004seconds (R₁=R′₁−R″ ₁), and the second region R′₂ in the frame 38 b willhave an exposure of 0.008 seconds.

FIG. 4B shows an alternative example that is similar to that shown inFIG. 4A, except that the pixels R″₂ of the second frame that are notwithin the specified region are stored in the memory rather than beingdiscarded. In this alternative example, steps 42-52 are the same asdescribed above with respect to FIG. 4A. However, in this alternative,the step 56 of discarding pixels R″₂ of the second frame that are notwithin the specified region is not performed. Instead, pixels R″₂ of thesecond frame that are not within the specified region are stored in thememory, replacing the pixels R′₂ of the frame that are not within thespecified region (step 54). Like the example of FIG. 4A, if the firstand second frames 30, 32 are both exposed at 0.004 seconds, then thefirst region R₁ in the frame 38 b will have an exposure, assuming theframes are combined by summing, of 0.008 seconds and the second regionR″₂ in the frame 38 b will have an exposure of 0.004 seconds.

FIG. 5 illustrates another example of creating a frame having exposuresthat vary by region. According to this alternative, the step 42 of themethod 40 does not store the full first frame. Instead, only thosepixels R′₁ of the first frame that are within the specified first regionare stored in the memory. Pixels R′₂ of the first frame are discarded.Steps 44-52 are the same as described above with respect to FIGS. 4A,4B. With respect to pixels of the second frame that are not found to bein the specified region, i.e., pixels R″₂, these pixels are stored inthe memory in step 54. In this alternative, the step 56 is notperformed. Like the examples of FIGS. 4A, 4B, if the first and secondframes 30, 32 are both exposed at 0.004 seconds, then the first regionR₁ will have an exposure of 0.008 seconds (R₁=R′₁+R″₁), and the secondregion will have an exposure of 0.004 seconds.

With respect to the flow diagram FIG. 3, it was stated above that thestep 42 of storing may store both the first and second regions of thefirst frame in the memory. It was also stated that the memory may havecapacity for storing less than one frame. FIG. 5 is one example of wherethe memory may have capacity for storing less than one frame. Inparticular, the memory for performing the step 42 need only be largeenough to store a first region of a frame. As mentioned, in the exampleof FIG. 5, pixels of the second frame that are not within the specifiedregion, i.e., the pixels R″₂, are stored in the memory in step 54.However, this memory need not be the same memory as is used to store instep 42 to store the pixels R′₁ of the first frame that are within thespecified first region. In other words, referring to FIG. 5, the pixelsR′₁ may be stored in a first memory and the pixels R₁=R′₁+R″₁ and R″₂may be stored in a second memory (or written to a device having a space34, e.g., a display device). The first memory may have capacity forstoring less than one frame while the second memory or device has thecapacity for storing or receiving a full frame.

FIG. 6 illustrates yet another example of creating a frame havingexposures that vary by region. In this alternative, the first and secondframes are captured with different exposures. For example, the firstframe 30 may be captured at an exposure of 0.008 seconds and the secondframe may be captured at an exposure of 0.004 seconds. In addition, inthis example, the step 42 does not store the full first frame. Instead,only those pixels R′₁ of the first frame that are within the specifiedfirst region are stored in the memory in step 42. Pixels R′₂ of thefirst frame are discarded. Steps 42-46 are the same as described abovewith respect to FIGS. 4A, 4B, and 5. However, replacement pixels are notgenerated (step 48) or stored in the memory (step 50). Pixels R″₁ of thesecond frame are discarded. With respect to pixels of the second framethat are not found to be in the specified region, i.e., pixels R″₂,these pixels are stored in the memory in step 54. In this alternative,the step 56 is generally not performed. Like the examples above, and thepixels R′₁ of the first region will have an exposure of 0.008 seconds,and the pixels R″₂ of the second region have an exposure of 0.004seconds.

FIG. 7 is a simplified block diagram of one embodiment of a displaysystem in which the exposure of a digital image may be varied by region.The system 60 may be a mobile device (defined below). Where the system60 is a mobile device, it is typically powered by a battery (not shown).The system 60 may include a display controller 62, a host 64, at leastone display device 66 and one or more image data sources, such as imagesensor 68.

Because the host 64 may be a source of image data, the term “image datasource” is intended to include the host 64. While the system 60 mayinclude multiple display devices and image data sources, this is notessential. In other embodiments, a single display device or a singleimage data source may be provided.

The display controller 62 interfaces the host 64 and image sensor 68with the display device 66. In the shown embodiment, the displaycontroller 62 is used to generate a digital image in which exposurevaries by region. In one embodiment, a position determining unit 70, acombining unit 72, and a region specifying unit 74 are provided in thedisplay controller for this purpose. In another embodiment, thecombining unit 72 may be omitted. It is not essential that the units 70,72, and 74 be provided in a display controller. The exposure of adigital image may be varied by region in a standalone unit or by otherunits in a system. In one embodiment, the display controller 62 is aseparate integrated circuit from the remaining elements of a system,that is, the display controller is “remote” from the host, image sensor,and display device.

The host 64 is typically a microprocessor, but it may be a digitalsignal processor, a computer, or any other type of device or machinethat may be used to control operations in a digital circuit. Typically,the host 64 controls operations by executing instructions that arestored in or on a machine-readable medium. The host 64 communicates withthe display controller 62 over a bus 76 to a host interface 78 in thedisplay controller 62. Other devices may be coupled with the bus 76. Forinstance, a memory 80 may be coupled with the bus 76. The memory 80 may,for example, store instructions or data for use by the host 64, or imagedata that may be rendered using the display controller 62. The memory 80may be an SRAM, DRAM, Flash, hard disk, optical disk, floppy disk, orany other type of memory.

A display device interface 82 is included in the display controller 62.The display device interface 82 provides an interface between thedisplay controller 62 and the display device 66. A display device bus 84couples the display controller 62 and the display device 66. LCDs aretypically used as display devices in mobile devices, but the displaydevice 66 (defined below) may be any type of display device.

The image sensor 68 may be, for example, a charge-coupled device (“CCD”)or a complementary metal-oxide semiconductor (“CMOS”) sensor. A camerainterface 86 (“CAM I/F”) is included in the display controller 62. Thecamera interface 86 is coupled with the image sensor 68 and receivespixel data output on data lines of a bus 88. Typically, the camerainterface 86 also receives vertical and horizontal synchronizing signalsfrom the image sensor 68 and provides a clocking signal to the imagesensor 68 for clocking pixel data out of the sensor. These signals maybe transmitted via the bus 88 or via a separate bus (not shown).

The position determining unit 70 receives image data from the imagesensor 68 via the camera interface 86, from the host 64 or memory 80 viathe host interface 78, or from any other desired image data source. Theposition determining unit 70 may provide image data to the combiningunit 72 or to a memory 90 for temporary storage before display. Thecombining unit 72 may also provide image data to the memory 90. Theregion specifying unit 74 is coupled with the position determining unit70. The position determining unit 70, the combining unit 72, and theregion specifying unit 74 are described in more detail below.

The memory 90 may be included in the display controller 62. In otherembodiments, however, the memory 90 may be remote from the displaycontroller. The memory 90 may be used as a frame buffer for storingimage data (and may be alternately referred to as a frame buffer orframe buffer memory), but may also be used for storing other types ofdata. For purposes of creating a digital image in which exposure variesby region, the memory 90 is of sufficient size to store one frame, butit is not necessary for the memory 90 to have capacity for storing morethan one frame. In one embodiment, the memory may have capacity forstoring just one region of a frame. The memory 90 is of the SRAM type,but the memory 90 may be a DRAM, Flash memory, hard disk, optical disk,floppy disk, or any other type of memory. The memory 90 may be coupledwith the position determining unit 70 and the combining unit 72, therebyallowing a digital image having exposure varying by region to be storedin the memory. The memory 90 may be coupled with other units within thegraphics controller 62 as necessary or desired.

A display pipe 92 is included in the display controller 62. The memory90 may be coupled with an input to the display pipe 92. An output of thedisplay pipe 92 may be coupled with the display interface 82. Thus, aframe in which exposure varies by region of the image may be transferredfrom the memory 90 to the display device 66 via the display pipe 92 anddisplay interface 82.

The position determining unit 70 may determine whether a pixel is of afirst or second frame. If a pixel is of the first frame, the unit 70 maywrite the pixel of the first frame to the memory 90 for storage. In oneembodiment, the unit 70 may write only those pixels of the first framethat are within a specified region to the memory 90 for storage. Theposition determining unit 70 may also determine whether a pixel that itreceives is within a specified region. If the entire first frame iswritten to the memory 90, e.g., the example of FIG. 4A, the positiondetermining unit 70 may not make this determination with respect topixels of the first frame. On the other hand, if the unit 70 writes onlythose pixels of the first frame that are within a specified region tothe memory 90, e.g., the example of FIG. 5, then the unit 70 makes thisdetermination for each first frame pixel.

With respect to pixels of the second region, the position determiningunit 70 may determine whether a pixel that it receives is within aspecified region. If the pixel is not within the specified region, theunit 70 may discard the pixel. Alternatively, the unit 70 may store thepixel in the memory 90 in a destructive write operation.

When the combining unit 72 writes a pixel of the first frame to thememory 90 it generally writes the pixel to a pixel location in thememory that corresponds with the coordinates of the pixel.

In addition, with respect to pixels of the second frame, if the pixel isdetermined to be within the specified region, the unit 70 may write thepixel to the combining unit 72. The combining unit 72 may then generatea replacement pixel by combining the pixel of the second frame that iswithin the specified region with the spatially corresponding pixel ofthe first frame. The combining unit 72 may read the spatiallycorresponding pixel of the first frame from the memory 90. The combiningunit 72 stores each of the replacement pixels in the memory 90 in pixellocations in the memory that correspond with the coordinates of thereplacement pixel. However, in one alternative, the position determiningunit 70 may discard a pixel of the second frame that is within thespecified region instead of creating and storing a replacement pixel.

The region specifying unit 74 provides the position determining unit 70with the locations of the first and second regions. In addition, theregion specifying unit 74 provides support for an interface that may beemployed for specifying a “specified” region. Regions 24, 26 may bespecified in a variety of ways. For instance, a user may specify aregion by inputting coordinates. The user may employ a stylus with atouch-sensitive screen to specify a region. In another alternative, auser may select from among several predetermined regions presented onthe display screen. In addition to selection by a user, the specifiedregion may be selected by an algorithm or a machine performing thealgorithm, where the algorithm selects regions based upon an analysis ofthe image data.

Various ways in which a digital image may be created in which exposurevaries by region have been described above by way of example. Theexamples refer only to first and second frames, and only to first andsecond regions. However, there is no intention to limit the presentinvention to only first and second frames, and only to first and secondregions. In alternative embodiments, an image in which exposure variesby region may be created using any number of regions, or using anynumber of frames. For instance, as one example, a first, second, andthird frame may be captured at an exposure of 0.003. The frames includefirst, second, and third regions. In addition, both the first and secondregions are “specified.” In this example, let all three regions of thefirst frame be written to memory at time T1. When the second frame isreceived at time T2, pixels in the first and second regions are storedin the memory in an additive write operation, while pixels in the thirdregion are discarded. When the third frame is received at time T3,pixels in the first region are stored in the memory in an additive writeoperation, while pixels in the second and third regions are discarded.After time T3, the first region of the frame stored in the memory willhave an exposure of 0.009 seconds, the second region will have anexposure of 0.006 seconds, and the third region will have an exposure of0.003 seconds.

In the above description of the various ways in which a digital imagemay be created in which exposure varies by region, the use of a memoryin this process was explained. It should be appreciated that thedescribed methods and apparatus may be employed with any type of devicehaving pixel locations defined in a space 34 and which may be writtento. As one example, a display device may be substituted for the memory90. A suitable display device may include an embedded memory and thus becapable of providing pixels of a first region of a first frame forcombining with spatially corresponding values of a second frame.Alternatively, the described methods and apparatus may be employed withdisplay device in a system that includes a memory sufficiently large forstoring only a first region, but which is of insufficient size forstoring a full frame, where the memory is used for reading pixels of afirst region of a first frame for combining with spatially correspondingvalues of a second frame. In another alternative, a display devicehaving a space 34 may be substituted for the memory 90 where there is noneed for reading pixels of a first region of a first frame for combiningwith spatially corresponding values of a second frame, such as shown inthe example of FIG. 6.

In this specification and in the claims, the terms “frame” and “image”are as used as synonyms.

The term “real-time,” as used in this specification and in the claims,refers to operations that are performed with respect to an external timeframe. More specifically, real-time refers to an operation or operationsthat are performed at the same rate or faster than a process external tothe machine or apparatus performing the operation. As an example, areal-time operation for generating a digital image in which exposurevaries by region of the image proceeds at the same rate or at a fasterrate than the rate at which pixels are received from an image sensor orother image data source, or the rate at which pixels are required by adisplay device or circuitry driving the display device. In variousembodiments of the present invention, a digital image in which exposurevaries by region may be generated in real-time.

Embodiments of the claimed inventions may be used in a “mobile device.”A mobile device, as the phrase is used in this description and theclaims, means a computer or communication system, such as a mobiletelephone, personal digital assistant, digital music player, digitalcamera, or other similar device. Embodiments of the claimed inventionsmay be employed in any device capable of processing image data,including but not limited to computer and communication systems anddevices generally.

The term “display device” is used in this description and the claims torefer to any of device capable of rendering images. For example, theterm display device is intended to include hardcopy devices, such asprinters and plotters. The term display device additionally refers toall types of display devices, such as CRT, LED, OLED, and plasmadevices, without regard to the particular display technology employed.

In this document, particular structures, processes, and operations wellknown to the person of ordinary skill in the art may not be described indetail in order to not obscure the description. As such, embodiments ofthe claimed inventions may be practiced even though such details are notdescribed. On the other hand, certain structures, processes, andoperations may be described in some detail even though such details maybe well known to the person of ordinary skill in the art. This may bedone, for example, for the benefit of the reader who may not be a personof ordinary skill in the art. Accordingly, embodiments of the claimedinventions may be practiced without some or all of the specific detailsthat are described.

In this document, references may be made to “one embodiment” or “anembodiment.” These references mean that a particular feature, structure,or characteristic described in connection with the embodiment isincluded in at least one embodiment of the claimed inventions. Thus, thephrases “in one embodiment” or “an embodiment” in various places are notnecessarily all referring to the same embodiment. Furthermore,particular features, structures, or characteristics may be combined inone or more embodiments.

Although embodiments have been described in some detail for purposes ofclarity of understanding, it will be apparent that certain changes andmodifications may be practiced within the scope of the appended claims.Accordingly, the described embodiments are to be considered asillustrative and not restrictive, and the claimed inventions are not tobe limited to the details given herein, but may be modified within thescope and equivalents of the appended claims. Further, the terms andexpressions which have been employed in the foregoing specification areused as terms of description and not of limitation, and there is nointention in the use of such terms and expressions to excludeequivalents of the features shown and described or portions thereof, itbeing recognized that the scope of the inventions are defined andlimited only by the claims which follow.

1. A hardware implemented method for creating a digital image in whichexposure varies by region of the image from two or more images, each ofthe pixels of the images having coordinates in a space, comprising: (a)writing pixels of a first one of the images that are within a firstregion of the space to pixel locations in a device that correspond withthe coordinates of the respective first image pixels; and (b) generatingreplacement pixels by combining each pixel of the first image that iswithin the first region with a spatially corresponding pixel in a secondone of the images, each replacement pixel having the same coordinates asthe pixels from which it is generated.
 2. The method of claim 1, whereinthe step (b) further comprises writing each of the replacement pixels toa pixel location in the device that corresponds with the coordinates ofthe replacement pixel.
 3. The method of claim 2, further comprising (c)writing pixels of a first one of the images that are within a secondregion of the space to pixel locations in the device that correspondwith the coordinates of the respective first image pixels.
 4. The methodof claim 3, further comprising (d) writing pixels of a second one of theimages that are within the second region to pixel locations in thedevice that correspond with the coordinates of the respective secondimage pixels, the step (d) being performed after the step (c).
 5. Themethod of claim 2, further comprising discarding pixels of a first oneof the images that are within a second region of the space.
 6. Themethod of claim 5, further comprising writing pixels of a second one ofthe images that are within the second region to pixel locations in thedevice that correspond with the coordinates of the respective secondimage pixels.
 7. The method of claim 1, wherein step (b) includes addinga first image pixel and a corresponding second image pixel.
 8. Themethod of claim 1, wherein step (b) includes subtracting one of a firstimage pixel and a corresponding second image pixel from the other. 9.The method of claim 1, wherein the device is a memory.
 10. A hardwareimplemented method for creating a digital image in which exposure variesby region of the image from two or more images, each of the pixels ofthe images having coordinates in a space, comprising: (a) writing pixelsof a first one of the images that are within a first region of the spaceto pixel locations in a device that correspond with the coordinates ofthe respective first image pixels; and (b) writing pixels of a secondone of the images that are within a second region of the space to pixellocations in the device that correspond with the coordinates of therespective second image pixels, wherein the second image has an exposureperiod that is distinct from the exposure period of the first image. 11.The method of claim 13, wherein the device is a memory.
 12. A displaycontroller comprising: (a) a memory to store image data, the memoryhaving a plurality of pixel locations for pixels of an image, the pixellocations corresponding with the coordinates of the respective pixels ina space; (b) a position determining unit to determine whether a pixel iswithin one of at least a first and a second region of the space; and (c)a combining unit to generate a replacement pixel by combining each pixelof a first image that is within the first region with a spatiallycorresponding pixel of a second image, the replacement pixel having thesame coordinates as the pixels from which it is generated, and to storethe replacement pixel in a pixel location in the memory that correspondswith the coordinates of the replacement pixel.
 13. The displaycontroller of claim 12, wherein the position determining unit stores afirst image pixel that is within the first region in a pixel location inthe memory that corresponds with the coordinates of the first imagepixel.
 14. The display controller of claim 13, wherein the combiningunit reads a first image pixel that is within the first region from thememory.
 15. The display controller of claim 12, wherein the positiondetermining unit stores a first image pixel that is within the secondregion in a pixel location in the memory that corresponds with thecoordinates of the first image pixel.
 16. The display controller ofclaim 12, wherein the position determining unit stores a second imagepixel that is within the second region in a pixel location in the memorythat corresponds with the coordinates of the second image pixel.
 17. Thedisplay controller of claim 12, wherein the combining unit combines afirst image pixel with a spatially corresponding second image pixel byadding the pixels.
 18. The display controller of claim 12, wherein thecombining unit combines a first image pixel with a spatiallycorresponding second image pixel by subtracting one pixel from theother.
 19. The display controller of claim 12, further comprising aregion specifying unit to specify the first and second regions.
 20. Thedisplay controller of claim 12, wherein the display controller generatesreplacement pixels in real-time.
 21. A display controller comprising:(a) a memory to store image data, the memory having a plurality of pixellocations for pixels of an image, the pixel locations corresponding withthe coordinates of the respective pixels in a space; and (b) a positiondetermining unit to determine whether a pixel is within one of at leasta first and a second region of the space, to write pixels of a firstimage that are within a first region to pixel locations in the memorythat correspond with the coordinates of the respective first imagepixels, and to write pixels of a second image that are within the secondregion to pixel locations in the memory that correspond with thecoordinates of the respective second image pixels, wherein the secondimage has an exposure period that is distinct from the exposure periodof the first image.